SoccerNet: A Scalable Dataset for Action Spotting in Soccer Videos

In this paper, we introduce SoccerNet, a benchmark for action spotting in soccer videos. The dataset is composed of 500 complete soccer games from six main European leagues, covering three seasons from 2014 to 2017 and a total duration of 764 hours. A total of 6,637 temporal annotations are automatically parsed from online match reports at a one minute resolution for three main classes of events (Goal, Yellow/Red Card, and Substitution). As such, the dataset is easily scalable. These annotations are manually refined to a one second resolution by anchoring them at a single timestamp following well-defined soccer rules. With an average of one event every 6.9 minutes, this dataset focuses on the problem of localizing very sparse events within long videos. We define the task of spotting as finding the anchors of soccer events in a video. Making use of recent developments in the realm of generic action recognition and detection in video, we provide strong baselines for detecting soccer events. We show that our best model for classifying temporal segments of length one minute reaches a mean Average Precision (mAP) of 67.8%. For the spotting task, our baseline reaches an Average-mAP of 49.7% for tolerances $\delta$ ranging from 5 to 60 seconds. Our dataset and models are available at https://silviogiancola.github.io/SoccerNet.Comment: CVPR Workshop on Computer Vision in Sports 201

An Empirical Study for PCA- and LDA-Based Feature Reduction for Gas Identification

Abstract: Increasing the number of sensors in a gas identification system generally improves its performance as this will add extra features for analysis. However, this affects the computational complexity, especially if the identification algorithm is to be implemented on a hardware platform. Therefore, feature reduction is required to extract the most important information from the sensors for processing. In this paper, linear discriminant analysis (LDA) and principal component analysis (PCA)-based feature reduction algorithms have been analyzed using the data obtained from two different types of gas sensors, i.e., seven commercial Figaro sensors and in-house fabricated 4×4 tin-oxide gas array sensor. A decision tree-based classifier is used to examine the performance of both the PCA and LDA approaches. The software implementation is carried out in MATLAB and the hardware implementation is performed using the Zynq system-on-chip (SoC) platform. It has been found that with the 4×4 array sensor, two discriminant functions (DF) of LDA provide 3.3% better classification than five PCA components, while for the seven Figaro sensors, two principal components and one DF show the same performances. The hardware implementation results on the programmable logic of the Zynq SoC shows that LDA outperforms PCA by using 50% less resources as well as by being 11% faster with a maximum running frequency of 122 MHz

Fast Prototyping of KNN Based Gas Discrimination System on the Zynq SoC

Electronic noses (EN) or machine olfaction are systems used for the detection and identification of odorous compounds and gas mixtures. The accuracy of such systems is as important as the processing time. Therefore, the choice of the algorithm and the implementation platform are both crucial. In this abstract, a design and implementation of a gas identification system on the Zynq platform which shows promising results is presented. The Zynq-7000 based platforms are increasingly being used in different applications including image and signal processing. The Zynq system on chip (SoC) architecture combines a processing system based on a dual core ARM Cortex processor with a programmable logic (PL) based on a Xilinx 7 series field programmable gate arrays (FPGAs). Using the Zynq platform, real-time hardware acceleration of classification algorithms can be performed on the PL and controlled by a software running on the ARM-based processing system (PS). The gas identification system is based on a 16-Array SnO2 in-house fabricated gas sensor and k-Nearest Neighbors (KNN) for classification. The KNN algorithm is executed on the PL for hardware acceleration. The implementation takes the form of an IP developed in C and synthesized using Vivado High Level Synthesis (HLS), the synthesis includes the conversion from C to register transfer level (RTL). The implementation requires the creation of a hardware design for the entire system that allows the execution of the IP on the PL and the remaining parts of the identification system on the PS. The hardware design is developed in Vivado using IP Integrator. The communication between the PS and PL is performed using advanced extensible interface protocol (AXI). A software application is written and executed on the ARM processor to control the hardware acceleration on the PL of the previously designed IP core and the board is programmed using Software Development Kit (SDK). An overview of the system architecture can be seen in Figure 1. The system is designed to discriminate five types of gases including C6H6, CH2O, CO, NO2 and SO2 at various concentrations, from 0.25 to 5 parts per million (ppm) for C6H6 and CH2O, from 5 to 200 ppm for CO, from 1 to 10 ppm for NO2 and finally from 1 to 25 ppm for SO2. The experimental setup used in the laboratory to collect the data is shown in Figure 2. It consists of a gas chamber where the sensor array is placed. The gas chamber has two orifices, one to serve as an input for the in-flow of gases and the other one as an exhaust to evacuate the gases. Multiple gases are stored in various cylinders and connected to the gas chamber individually through several Mass Flow Controllers (MFCs). A control unit is connected to the MFCs to control the in-flow of gases and to the sensor array via a Data Acquisition (DAQ) system to collect and sample the response of the sensor array. In total, 192 samples are collected, 50% is used for training and the other 50% is used for testing. Simulations were performed in MATLAB environment prior to the implementation on the hardware where different k values have been used. The Euclidean distance has been used as a metric for the computation of distances between various points. The best results were obtained for k = 1 and k = 2 with a classification accuracy of 97.91% and 98.95% respectively. The system implemented on hardware is based on k = 1 since the accuracies are almost similar while the hardware resources required for k = 2 are much higher than for k = 1. This can be explained by the fact that in the case of k = 2 we need to sort the vector of distances to be able to find the nearest two neighbours while in k = 1 we only need to find the smallest distance. The target hardware implementation platform of the proposed KNN is the heterogeneous Zynq SoC. The implementation is based on the use of Vivado HLS. A summary of the design flow is presented in Figure 3. The starting point is Vivado HLS where the KNN block is converted from C/C++ implementation to a RTL based IP core. This allows a considerable gain in development time without scarifying on high parallelism characteristics because Vivado HLS provides a large number of powerful optimization directives. The generated IP-core is then exported and stored in the Xilinx IP Catalog before being used in Vivado IP Integrator to create the hardware block design with all needed components and interconnections. The next step is to export the generated hardware along with IP drivers to the SDK tool. The SDK tool is used to program the Xilinx ZC702 prototyping board via joint test action group (JTAG) interface and the terminal in SDK is used to communicate with the board via universal asynchronous receiver/transmitter (UART) interface. The KNN IP is implemented on the PL of the Zynq SoC and communicates with the PS part via the Xilinx AXI-Interconnect IP. A software is written in C/C++ and executed on the PS to manage the IP present in the PL in terms of sending the input data, waiting for the interrupt and then reading the output data. The block design and the resulting chip layout are shown in Figure 4. It is worth mentioning that the running frequency for the ARM processor is set to the maximum 667MHz while the PL frequency is set to 100 MHz which is the maximum for the KNN IP generated in HLS. The real execution of KNN on the PL side of the ZC702 board shows that one sample can be processed for gas identification in 0.0078 ms while the same sample requires 0.9228 ms if executes on the PS side in the ARM processor in a pure software manner. This means that a speed up of 118 times has been achieved. The main directive in Vivado HLS that helped to reach these performances is the "Loop pipelining" which allows the operations in a loop to be implemented in a concurrent manner. The hardware resources usage can be seen in Figure 5, it shows that 24% of lookup tables (LUT), 12% of flip-flops (FF), 6% of BRAM and 58% DSP have been used. As shown in Figure 6, the total power consumption is 1.895 W, 1.565 W is consumed by the PS and the remaining 0.33W is consumed by the PL.qscienc

Continuous-Time ΣΔ ADC with Implicit Variable Gain Amplifier for CMOS Image Sensor

This paper presents a column-parallel continuous-time sigma delta (CTSD) ADC for mega-pixel resolution CMOS image sensor (CIS). The sigma delta modulator is implemented with a 2nd order resistor/capacitor-based loop filter. The first integrator uses a conventional operational transconductance amplifier (OTA), for the concern of a high power noise rejection. The second integrator is realized with a single-ended inverter-based amplifier, instead of a standard OTA. As a result, the power consumption is reduced, without sacrificing the noise performance. Moreover, the variable gain amplifier in the traditional column-parallel read-out circuit is merged into the front-end of the CTSD modulator. By programming the input resistance, the amplitude range of the input current can be tuned with 8 scales, which is equivalent to a traditional 2-bit preamplification function without consuming extra power and chip area. The test chip prototype is fabricated using 0.18 m CMOS process and the measurement result shows an ADC power consumption lower than 63.5 W under 1.4 V power supply and 50 MHz clock frequency

A Low Power Reconfigurable Multi-sensing Platform For Gas Application

Presence of toxic gases and accidental explosions in gas industries have turned the researcher to innovate an electronic nose system which can indicate the nature and the parameters of the gas passing through different vessels. Therefore, in this research we propose a low power Radio Frequency Identification (RFID) based gas sensor tag which can monitor the parameters and indicate the type of gas. The research work is divided in to three main parts. The first two parts cover the design and analysis of low power multi-sensors and processing unit, while the last part focuses on a passive RFID module which can provide communication between the sensor and the processing unit, as shown in Fig. 1. In passive RFID applications, power consumption is one of the most prominent parameter because most of the power is harvested from the coming RF signal. Therefore a ring-oscillator based low power temperature sensor is designed to measure the gas thermodynamic conditions. The oscillator is designed using the Thyristor based delay element [7], in which the current source present for temperature compensation has been displaced to make the delay element as temperature dependent. The proposed temperature sensor consumes 47nW power at 27 °C, which increases linearly with temperature. Moreover, a 4x4 array of tin-oxide gas sensor based on convex Micro hotplates (MHP), is also utilized to identify the type of gas. The array is designed such that each sensor of an array provide different pattern for the same gas. The power consumption caused by the temperature and gas sensor is in the order of few µW's. The prime advantage of MHP can be visualized by the 950 °C annealed MHP, which exhibit the thermal efficiency of 13 °C /mW. Moreover it requires a driving voltage of only 2.8V to reach 300 °C in less than 5ms, which make it compatible with power supplies required by CMOS ICs. The gas sensor will provide 16 feature points at a time, which can results in hardware complexity and throughput degradation of the processing unit. Therefore, a principle component analysis (PCA) algorithm is implemented to reduce the number of feature points. Thereafter, a binary decision tree algorithm is adopted to classify the gases. We implemented both algorithms on heterogeneous Zynq platform. It is observed that the execution of PCA on Zynq programmable SoC is 1.41 times faster than the corresponding software execution, with a resource utilization of only 23% . Finally, a passive ultrahigh-frequency (UHF) RFID transponder is developed for communicating between the sensing block and processing unit. The designed module is responsible to harvest the power from the coming RF signal and accomplish the power requirement of both sensors. The designed transponder IC achieved minimum sensitivity of -17dBm with a minimum operational power of 2.6µW.qscienc

Embedded Platform for Gas Applications Using Hardware/Software Co-Design and RFID

This paper presents the development of a wireless low power reconfigurable self-calibrated multi-sensing platform for gas sensing applications. The proposed electronic nose (EN) system monitors gas temperatures, concentrations and mixtures wirelessly using the radio-frequency identification (RFID) technology. The EN takes the form of a set of gas and temperature sensors as well as multiple pattern recognition algorithms implemented on the Zynq system on chip (SoC) platform. The gas and temperature sensors are integrated on a semi-passive RFID tag to reduce the consumed power. Various gas sensors are tested including an in-house fabricated 4×4 SnO2 based sensor and 7 commercial Figaro sensors. The Data is transmitted to the Zynq based processing unit using a RFID reader where it is processed using multiple pattern recognition algorithms for dimensionality reduction and classification. Multiple algorithms are explored for optimum performance including principal component analysis (PCA) and linear discriminant analysis (LDA) for dimensionality reduction while decision tree (DT) and k-nearest neighbors (KNN) are assessed for classification purpose. Different gases are targeted at diverse concentration including carbon monoxide (CO), ethanol (C2H6O), carbon dioxide (CO2), propane (C3H8), ammonia (NH3) and hydrogen (H2). An accuracy of 100% is achieved in many cases with an overall accuracy above 90% in most scenarios. Finally, the hardware/software heterogeneous solution to implementation PCA, LDA, DT and KNN on the Zynq SoC shows promising results in terms of resources usage, power consumption and processing time

HW/SW co-design based implementation of Gas discrimination

A gas discrimination system is mainly made of two parts, the sensing part and the processing part. As an alternative solution to pure software or hardware implementation of the processing part of a gas identification system, this paper proposes a gas discrimination system and its implementation on the Zynq system on chip platform using hardware/software co-design approach. In addition, the recommended system uses principal component analysis for dimensionality reduction, binary decision tree for classification and a 4�4 in-house gas sensor array for sensing. Moreover, k-nearest neighbors classifier is also used and compared with decision tree. MATLAB is used for simulation and validation before the final implementation on the Zynq. Algorithms are implemented using high level synthesis and different optimization directives are applied. Hardware implementation results on the Zynq show that real-time performances can be achieved for proposed e-nose system using hardware/software co-design approach with a single ARM processor running at 667 MHz and the programmable logic running at 142 MHz.Scopu

Electronic nose system on the Zynq SoC platform

Electronic nose or machine olfaction are systems used for detection and identification of odorous com- pounds and gas mixtures. An electronic nose system is mainly made of two parts, the sensing part which takes the form of a single or a set of sensors and the processing part which takes the form of some pat- tern recognition algorithms. As an alternative solution to pure software or hardware implementation of the processing part of a gas identification system, this paper proposes a hardware/software co-design ap- proach using the Zynq platform for the implementation of an electronic nose system based on principal component analysis as a dimensionality reduction technique and decision tree as a classification algo- rithm using two different sensors array, a 4 ×4 in-house fabricated sensor and a commercial one based on 7 Figaro sensors, for comparison purpose. The system was successfully trained and simulated in MAT- LAB environment prior to the implementation on the Zynq platform. Various scenarios were explored and discussed including the investigation of different combination of principal components as well as the uti- lization of drift compensation technique to improve the identification accuracy. High level synthesis was carried out on the proposed designs using different optimization directives including loop unrolling, ar- ray partitioning and pipelining. Hardware implementation results on the Zynq system on chip show that real-time performances can be achieved for proposed electronic nose systems using hardware/software co-design approach with a single ARM processor running at 667 MHz and the programmable logic run- ning at 142 MHz. In addition, using the designed IP cores and for the best scenarios, a gas can be iden- tified in 3.46 μs using the 4 ×4 sensor and 0.55 μs using the Figaro sensors. Furthermore, it has been noticed that the choice of the sensor array has an important impact on performances in terms of ac- curacy and processing time. Finally, it has been demonstrated that the programmable logic of the Zynq platform consumes much less power than the processing system.This paper was made possible by National Priorities Research Program (NPRP) grant No. 5 - 080 - 2 - 028 from the Qatar National Research Fund (a member of Qatar Foundation). The statements made herein are solely the responsibility of the authors.Scopu

Continuous-Time Sigma Delta ADC with Implicit Variable Gain Amplifier for CMOS Image Sensor

This paper presents a column-parallel continuous-time sigma delta (CTSD) ADC for mega-pixel resolution CMOS image sensor (CIS). The sigma delta modulator is implemented with a 2nd order resistor/capacitor-based loop filter. The first integrator uses a conventional operational transconductance amplifier (OTA), for the concern of a high power noise rejection. The second integrator is realized with a single-ended inverter-based amplifier, instead of a standard OTA. As a result, the power consumption is reduced, without sacrificing the noise performance. Moreover, the variable gain amplifier in the traditional column-parallel read-out circuit is merged into the front-end of the CTSD modulator. By programming the input resistance, the amplitude range of the input current can be tuned with 8 scales, which is equivalent to a traditional 2-bit preamplification function without consuming extra power and chip area. The test chip prototype is fabricated using 0.18 mu m CMOS process and the measurement result shows an ADC power consumption lower than 63.5 mu W under 1.4 V power supply and 50 MHz clock frequency